Three-dimensional tracking of a target in a body

ABSTRACT

Disclosed is a method and system for three-dimensional tracking of a target located within a body, the method performed using at least one processing system. A two-dimensional scanned image of the body including the target is processed to obtain a two-dimensional image of the target. A first present dataset of the target is predicted using a previous dataset of the target and a state transition model, the first present dataset includes a three-dimensional present position value of the target. A second present dataset of the target is measured by template-matching of the two-dimensional image of the target with a model of the target. A third present dataset of the target is estimated by statistical inference using the first present dataset and the second present dataset. The previous dataset of the target is updated to match the third present dataset.

TECHNICAL FIELD

The present invention generally relates to a system and/or a method orprocess for tracking a target, and more specifically to a system and/ora method or process for three-dimensional tracking of a target locatedwithin a body. In further specific examples, a system and/or a method orprocess is formed or provided for markerless tracking of a target inthree dimensions, using two-dimensional images of the target.

BACKGROUND

Lung tumour motions are clinically significant and unpredictable. Duringlung cancer radiotherapy, tumour motion is a major challenge to accuratebeam targeting of tumours. There is currently no technology able todirectly track tumour motion in three dimensions during treatment.

Current motion management techniques rely on pre-treatment imagingtechnologies such as four-dimensional computed tomography (4D-CT) andfour-dimensional cone-beam computed tomography (4D-CBCT), which may notactually represent the actual tumour motion during treatment.4D-CT-based margins have been known to underestimate lung tumour motionand to lead to significant tumour underdose for lung proton therapy.

Current real-time tumour tracking technologies rely on the implantationof radiopaque fiducial markers or electromagnetic transponder beacons.However, marker or beacon implantation is an invasive and costlyprocedure that is not widely available. Marker-induced toxicity andsurrogacy errors are also common problems.

Markerless tumour tracking methods have been proposed using kilovoltage(kV) and megavoltage (MV) imaging. A major challenge of markerlesstumour tracking is that tumours are rarely consistently visible on kV orMV projections due to obstruction by surrounding anatomic structures andchanges in radiological depth due to gantry rotation. Obstruction by thetreatment beam aperture in intensity-modulated radiation therapy (IMRT)treatments is also a common problem for MV-based methods.

Presently, there is insufficient tumour visibility for tracking tumoursin patients using the anterior-posterior (AP) view. Other views havelarger radiological depths and are generally more challenging fortracking than the AP view. Consequently, more patients are expected tobe ineligible for markerless tracking during gantry rotation. Anotherlimitation is the lack of ground truth for evaluating the accuracy ofmarkerless tumour tracking on clinical data. Most published studiesvalidated their tracking results by correlating the tracked trajectorieswith surrogate signals (e.g. abdominal or diaphragm motion) or comparingwith subjective measurements of tumour positions (e.g. visual inspectionand manual segmentation).

Additional limitations of X-ray imaging techniques include the fact thatX-ray imaging only provides two-dimensional (2D) imaging information.Motion information parallel to the X-ray beam path is not available. Forthis reason, tumour tracking using X-ray imaging suffers from largetracking uncertainties.

There is a need for new or improved systems for tracking tumour motionsand/or methods or processes of tracking tumour motions.

The reference in this specification to any prior publication (orinformation derived from the prior publication), or to any matter whichis known, is not, and should not be taken as an acknowledgment oradmission or any form of suggestion that the prior publication (orinformation derived from the prior publication) or known matter formspart of the common general knowledge in the field of endeavour to whichthis specification relates.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the PreferredEmbodiments. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

According to one example aspect, there is provided a method forthree-dimensional tracking of a target located within a body, the methodperformed by at least one processing system, and comprising the stepsof: processing a two-dimensional scanned image of the body including thetarget to obtain a two-dimensional image of the target, wherein thetwo-dimensional scanned image is acquired from a scanning device;predicting a first present dataset of the target by using a previousdataset of the target and a state transition model, wherein the firstpresent dataset includes at least a three-dimensional present positionvalue of the target, and wherein the previous dataset includes at leasta three-dimensional previous position value of the target; measuring asecond present dataset of the target by template-matching of thetwo-dimensional image of the target with a model of the target, whereinthe second present dataset includes at least a two-dimensional presentposition value of the target; estimating a third present dataset of thetarget by statistical inference using the first present dataset and thesecond present dataset, wherein the third present dataset includes atleast a three-dimensional present position value of the target; andupdating the previous dataset of the target to match the third presentdataset.

According to another example aspect, there is provided a system forthree-dimensional tracking of a target located within a body,comprising: a scanning device configured to acquire a two-dimensionalscanned image of the body including the target; and a processing systemconfigured to: receive and process the two-dimensional scanned image ofthe body including the target to obtain a two-dimensional image of thetarget; predict a first present dataset of the target by using aprevious dataset of the target and a state transition model, wherein thefirst present dataset includes at least a three-dimensional presentposition value of the target, and wherein the previous dataset includesat least a three-dimensional previous position value of the target;measure a second present dataset of the target by template-matching ofthe two-dimensional image of the target with a model of the target,wherein the second present dataset includes at least a two-dimensionalpresent position value of the target; estimate a third present datasetof the target by statistical inference using the first present datasetand the second present dataset, wherein the third present datasetincludes at least a three-dimensional present position value of thetarget; and update the previous dataset of the target to match the thirdpresent dataset.

BRIEF DESCRIPTION OF FIGURES

Example embodiments are apparent from the following description, whichis given by way of example only, of at least one preferred butnon-limiting embodiment, described in connection with the accompanyingfigures.

FIG. 1 illustrates an example method for three-dimensional tracking of atarget located within a body.

FIG. 2 illustrates an example implementation of the method of FIG. 1,where the target is a lung tumour.

FIG. 3 illustrates an example method for providing a model of the targetand a model of the body anatomy.

FIG. 4 illustrates an example method for acquiring a two-dimensionalimage of the target.

FIG. 5 illustrates an example system for three-dimensional tracking of atarget located within a body.

FIG. 6 illustrates an example processing system for use in the system ofFIG. 5.

FIG. 7 illustrates an example system for tracking and interacting with atarget located within a body.

FIG. 8 illustrates an example radiotherapy system for tracking andtreating a tumour located within a body.

FIG. 9 illustrates example tumour trajectories tracked using the methodof FIG. 1. Also illustrated are the example tumour trajectories trackedusing a conventional marker-based method, for comparison.

FIG. 10 illustrates further example tumour trajectories tracked usingthe method of FIG. 1. Also illustrated are the example tumourtrajectories tracked using a conventional marker-based method, forcomparison.

FIG. 11 illustrates mean tracking errors in the left-right (LR),superior-inferior (SI), and anterior-posterior (AP) directions for 13example cases investigated using the method of FIG. 1.

FIG. 12 illustrates a comparison between the markerless tumour trackingmethod and the standard of care in terms of the margins required toencompass 95% of tumour motion plotted with respect to the 5th-to-95thmotion ranges in the LR, AP, and SI directions, for the 13 example casesinvestigated using the method of FIG. 1.

PREFERRED EMBODIMENTS

The following modes, given by way of example only, are described inorder to provide a more precise understanding of the subject matter of apreferred embodiment or embodiments. In the figures, incorporated toillustrate features of an example embodiment, like reference numeralsare used to identify like parts throughout the figures.

In one broad form, there is provided a method for tracking a targetlocated within a body. The method includes acquiring a model of thetarget and a model of the body anatomy excluding the target. Then,acquiring a projected image of the target. Then, predicting a firstpresent dataset of the target by using a previous dataset of the targetand a state transition model. The method further includes measuring asecond present dataset of the target by template-matching of theprojected image with the model of the target. Then, estimating a thirdpresent dataset of the target by statistical inference using the firstpresent dataset and the second present dataset. Then, updating theprevious dataset. Each dataset of the target (i.e. first, second, andthird) includes a position value and a measure of the uncertainty of theposition value of the target.

Referring to FIG. 1, there is illustrated an example method 100 forthree-dimensional tracking of a target located within a body, method 100being performed by at least one processing system. Method 100 includesstep 110 of processing a two-dimensional scanned image of the bodyincluding the target to obtain a two-dimensional image of the target,wherein the two-dimensional scanned image is acquired from a scanningdevice. At step 120, predicting a first present dataset of the target byusing a previous dataset of the target and a state transition model,wherein the first present dataset includes at least a three-dimensionalpresent position value of the target, and wherein the previous datasetincludes at least a three-dimensional previous position value of thetarget. At step 130, measuring a second present dataset of the target bytemplate-matching of the two-dimensional image of the target with amodel of the target, wherein the second present dataset includes atleast a two-dimensional present position value of the target. At step140, estimating a third present dataset of the target by statisticalinference using the first present dataset and the second presentdataset, wherein the third present dataset includes at least athree-dimensional present position value of the target. Then, at step150, updating the previous dataset of the target to match the thirdpresent dataset.

An example of a target is a tumour. Other examples of a target includean abnormal growth of tissue, a biological organ, or biological tissue.An example of a body is a human body. Another example of a body is ananimal body.

In some examples, the steps of method 100 are executed sequentially. Insome examples, at least some steps of method 100 are executed inparallel.

A dataset of the target describes a state of the target, which mayinclude information on the position of the target and may furtherinclude statistical information on the position of the target. Forexample, the dataset may comprise a position value and a covariancevalue as a measure of the uncertainty of the position value. Thestatistical information may be obtained from present or past trackinginformation, present or past measurements, and/or any other knowledge onthe nature of the target and of the tracking process/measurement system(e.g. random and/or systematic errors of the measurement system).

In some examples, each dataset (i.e. first, second, and third datasets)further includes a measure of an uncertainty in the respective positionvalue of the target (i.e. the two- or three-dimensional position valueincluded in the dataset).

Preferably, though not necessarily, the position value in a dataset ofthe target refers to a centroid position of the target. The positionvalue may be relative to the body, or relative to a portion of the body,or relative to some other coordinate system defined with reference toany point internal or external to the body.

In some examples, the statistical inference utilises Bayesianprobability, or a Bayesian probability approach. In some examples, thestatistical inference utilises an extended Kalman filter, or an extendedKalman filter approach.

Preferably, though not necessarily, method 100 implements a Bayesianframework based on the extended Kalman filter. The Kalman filter, aswell as its extended version, is a statistical inference approach forupdating a state variable and the covariance matrix of its distribution,by combining a prediction and a measurement of the state variable. Themeasurement of the state variable contains some inherent uncertainty(e.g. due to noise or due to physical limitations).

In some example embodiments of method 100, the target is tracked inthree-dimensional (3D) space. In such examples, the state variable, x,represents the 3D position of the target centroid, and the covariancematrix, P, describes the distribution of the target position in 3Dspace. Since step 130 is effectively measuring a two-dimensionalposition of the target from a two-dimensional image of the target,method 100 requires extrapolation of the target's position in 3D spacefrom a 2D image. The power of a Kalman filter for statistical inferencein step 140 is that covariance matrix P exploits both the uncertaintiesand the correlations of each motion component, making it possible toestimate the 3D position of the target based on a 2D measurement.

In some examples, the target is a surrogate object, such as a radiopaquemarker or a beacon implanted in a human body. In other examples, thetarget is a soft tissue target. In some examples, the target is atumour. In some examples, the target is a lung tumour, moving due to apatient's breathing pattern.

Tracking of a moving target (such as a lung tumour) may require repeatedexecution of certain steps of method 100. In some examples, acquisitionof a two-dimensional scanned image is repeated, or continuouslyexecuted, at multiple points in time, and steps 110, 120, 130, 140, and150 are executed after each acquisition of the two-dimensional scannedimage, to continuously track the target's position. In some examples,tracking occurs in real time, for example, during treatment such as aradiotherapy session.

Method 100 may further include a step of adjusting an apparatusoperating on, or interacting with, the target in response to the thirdpresent dataset. In some examples where the target is a tumour, theapparatus may be a treatment apparatus, such as a radiation source forradiotherapy, or a patient table. This apparatus should be adjusted(e.g. by varying the direction of a radiation beam) in response to themotion of the tumour. The third present dataset estimated by method 100provides information on such motion. Therefore, the apparatus may beadjusted (either manually or automatically) following each execution ofmethod 100, as the position of the target is tracked.

Referring to FIG. 2, there is illustrated an example embodiment ofmethod 100 where the target is a lung tumour. FIG. 2 further illustratesadditional step 105, of acquiring a model of the target and a model ofthe body anatomy excluding the target. In some examples, step 105 may becomputationally intensive and as such, it is advantageous that it not berepeated prior to each iteration of method 100. In some examples, step105 is executed once for each patient, before commencement of atreatment. The acquired models of the target and of the body anatomy canthen be stored on a processing system, or any memory device. Executionof method 100 may then commence, with the models of the target and ofthe body anatomy being accessed as required during execution of method100.

The model of the target and/or the model of the body anatomy provideinformation of the spatial distribution, or location, of the target andof the body anatomy at one or more points in time, or at one or moreinstances in a time period. The time period may span one or more cyclesof a physiological process, such as a respiratory cycle. Preferably,though not necessarily, the spatial distribution information provided bythe models is 3D spatial distribution information. The model of the bodyanatomy may provide information (e.g. location, distribution,arrangement, and motion information) on the internal structure andcomponents of the body that contains the target.

In some examples, the model of the target and/or the model of the bodyanatomy is any one of, or a combination of: a physical model, ascientific model, a 3D model, and a computer model. The model of thetarget and/or the model of the body anatomy may include one or morevectors, matrices, and/or data structures.

In one example, the model of the target and/or the model of the bodyanatomy are derived, or built, from one or more images. These images maybe acquired prior to a treatment (e.g. radiation treatment) and aretherefore referred to as “prior images”.

In some examples, the model of the target is derived from prior imagesand from a surrogate signal. The surrogate signal may be indicative of amotion of the body and/or of the target. The surrogate signal may beused to augment, or enhance, the model of the target by providingfurther information on the target's motion. The surrogate signal may beprovided, or emitted, from a beacon or probe located internally orexternally to the body. In some examples, the surrogate signal tracks anexternal motion of the body. In some examples, the surrogate signal isestimated, detected, or measured from the projected image. The surrogatesignal may be emitted by a surrogate object, such as a radiopaque markeror a beacon implanted in the body. In some examples, method 100 furthercomprises a step of implanting a surrogate object in the body prior toacquiring the 2D scanned image (i.e. prior to step 110).

In one example, the prior images are 3D images. In another example, theprior images are 3D diagnostic images of a patient. In another example,the prior images span different points in time such that their subjectis illustrated in each prior image at a different point in time. In apreferred example, the prior images are four-dimensional cone beamcomputed tomography (4D-CBCT) images.

In some examples, the prior images are X-ray images. The prior imagespreferably illustrate the target and its neighbouring region within thebody (including anatomical features of the neighbouring regions withinthe body). A high-quality prior image is significant as it providesknowledge of how the target can be differentiated from surroundingobjects.

Referring to FIG. 3, there is illustrated an example method 300 foracquiring the model of the target and the model of the body anatomy.Method 300 includes step 310 of acquiring one or more prior images ofthe body anatomy including the target. Then, at step 320, identifyingportions within the prior images relating to the target. Then, at step330, forming the model of the target with the identified portions. Then,at step 340, modifying the prior images to make the identified portionstransparent. Then, at step 350, forming the model of the body anatomy,excluding the target, with the modified prior images.

Referring to FIG. 2, there is illustrated an example embodiment of step105, where a lung tumour (i.e. target) model 220 and a body anatomymodel 230 of a patient are built from a pre-treatment 4D-CBCT image 210.This is done by separating, or extracting, the tumour image from the4D-CBCT image. The resulting 4D tumour image is tumour model 220, whilethe 4D anatomic body image (without the tumour) is anatomic body model210.

In one example, the process of separating the tumour image from the4D-CBCT image is done by first warping a gross tumour volume (GTV)contour onto the 4D-CBCT image using deformable image restoration. TheGTV contour may be acquired by contouring the tumour on a planning CT.For each phase of the 4D-CBCT, the image pixels within the warpedcontour are exported, or extracted, to form a tumour image. Similarly,for each phase of the 4D-CBCT, the values of the image pixels within thewarped contour are set to zero attenuation, to form a “tumour-removed”body anatomy image. Multiple phases of the tumour images and the bodyanatomy images forms are used to generate 4D tumour images and 4Danatomic body images, and hence the tumour model and the body anatomymodel, respectively.

The qualities of both models are highly dependent on the quality of the4D-CBCT prior images. Different methodologies can be utilised to acquirethe prior image. For example, a 4D-CBCT prior image may be constructedusing the anatomical-adaptive image regularisation technique of Shieh etal (Shieh C C, Kipritidis J, O'Brien R T, Cooper B J, Kuncic Z and KeallP J 2015 Improving thoracic four-dimensional cone-beam CT reconstructionwith anatomical-adaptive image regularization (AAIR) Phys. Med. Biol.60(2), 841) to reduce noise and streaking artifacts while preservingimage sharpness. A prior-image-constrained-compressed-sensing (PICCS)algorithm, such as that provided by Chen et al (Chen G H, Tang J andLeng S 2008 Prior image constrained compressed sensing (PICCS): A methodto accurately reconstruct dynamic CT images from highly undersampledprojection data sets Med. Phys. 35(2), 660-663), may further be used toimprove the contrast of a bony anatomy.

In another example, sets of tomographic images of the patient that aretypically acquired for patient setup immediately before radiotherapytreatment, are suitable prior images because they very closely representthe patient anatomy during the treatment.

More details on the steps of method 100 as illustrated in FIG. 2 areprovided below.

Step 110

Referring to FIG. 4, there is illustrated an example method forprocessing a two-dimensional scanned image of the body including thetarget to obtain a two-dimensional image of the target. Method 400includes step 410 of acquiring a two-dimensional scanned image of thebody including the target. The two-dimensional scanned image of the bodyincluding the target preferably illustrates anatomical features of thebody (i.e. body anatomy) in the vicinity of the target. Then, at step420, projecting the model of the body anatomy to align with the bodyanatomy of the two-dimensional scanned image. Then, at step 430,subtracting the projected model of the body anatomy from thetwo-dimensional scanned image.

In some examples, the two-dimensional scanned image of the bodyincluding the target is acquired by a diagnostic medical imagingmodality. In some examples, the scanning device is a diagnostic medicalimaging device. In some examples, the two-dimensional scanned imagefurther includes a surrogate signal. In some examples, thetwo-dimensional scanned image is an X-ray image projection. In morespecific examples, the two-dimensional scanned image is a kV X-ray imageprojection.

A short imaging arc may be used to acquire the kV projection for atwo-dimensional scanned image. In some examples, an arc size of ninedegrees may be chosen. Experiments have found that nine degrees is thesmallest arc size that renders successful tracking. Compared to using asingle projection, a nine-degree arc exploits the 3D information of thetumour in multiple views, making it possible to track cases that wouldotherwise be challenging, e.g. tumours attached to neighbouringstructures. In addition, a nine-degree imaging arc can be acquired in1.5-9 seconds with a typical gantry speed of 1-6 deg/s, which is ingeneral a sufficiently short time interval for the time resolutionrequired for treatment guidance. A larger arc of thirty degrees wastested and found to slightly improve tumour localisation, but at thesame time further degrade the time resolution, leading to overallsimilar tracking performance. In practice, the optimal arc size maydepend on multiple factors such as the visibility, size, and location ofthe tumour, and gantry speed.

In some examples, method 400 is at least partially implemented, or run,through a graphics processing unit (GPU).

Referring to FIG. 2, there is illustrated an example embodiment of step110, showing a two-dimensional scanned image which is a kV projection240 of the body anatomy including the lung tumour (i.e. target). Alsoshown is a projection, or forward-projection, 250 of the model of thebody anatomy to align with the body anatomy in kV projection 240.

For acquiring model projection 250, the respiratory phase in kVprojection 240 is first determined, for example, by using the projectionintensity analysis method of Kavanagh et al (Kavanagh A, Evans P M,Hansen V N and Webb S 2009 Obtaining breathing patterns from anysequential thoracic x-ray image set Phys. Med. Biol. 54(16), 4879). Inother examples, the real-time phase can be calculated using the methodproposed by Ruan et al (Ruan D, Fessler J A, Balter J M and Keall P J2009 Real-time profiling of respiratory motion: baseline drift,frequency variation and fundamental pattern change Phys. Med. Biol.54(15), 4777-4792). Preferably, the body anatomy model used has the samephase as the phase of kV projection 240. For example, where the bodyanatomy model is generated using 4D-CBCT, an image having the same phaseas kV projection 240 may be selected for producing model projection 250.In the example embodiment of FIG. 2, this is done by forward projectingthe tumour-removed 4D-CBCT image of the same phase as kV projection 240to generate tumour-removed digitally reconstructed radiographs (DRRs) atthe gantry angles used to acquire kV projection 240.

Finally, a two-dimensional, or projected, image 260 of the tumour isgenerated by subtracting model projection 250 (i.e. the DRR image) fromkV projection 240. The difference projection thus acquired is assumed tocontain only attenuation contributed from the tumour, therefore thetumour position can be measured by matching the tumour model withtwo-dimensional image 260. In practice however, exact subtraction ofanatomies from the projections might not possible, or might bedifficult, due to the change in body anatomy during treatment and due toapproximation errors from the reconstructed 4D-CBCT images and DRRs.

Step 120

Prediction step 120 estimates the likely tumour position prior tomeasurement using a state transition model. The state transition modeldescribes how the tumour position is likely to evolve from a previousframe (k−1) to a current frame k based on some prior knowledge of thenature of the motion to be tracked. The state transition model may bederived from the model of the target, from the model of the bodyanatomy, and/or from any other information that is available regardingthe target and the body (e.g. information from a surrogate signal).

Where information on the motion of the target is available (e.g. throughprior images and/or a surrogate signal) prediction step 120 may takethis information into account to provide a more accurate estimate of thefirst present dataset of the target. Therefore, the motion informationmay be incorporated in the model of the target or in the statetransition model.

In cases where the target is a lung tumour, in some examples the modelof the target accounts for the periodic (or quasiperiodic) nature oflung tumour motion (e.g. due to respiration). In some examples,predicting the first present dataset of the target further comprisesaccounting for the periodic nature of lung tumour motion.

Referring to FIG. 2, there is illustrated an example embodiment ofprediction step 120. For lung tumour motion, which is periodic orquasiperiodic due to respiration, the state transition model can bebuilt using the 4D-CBCT prior image and the respiratory phases of the(k−1)'th and k'th frame. The model of the target and/or the model of thebody anatomy may be used to calculate the displacements in tumourposition from one phase bin l to another phase bin m, d_(l) ^(m). Insome examples, the prior images, such as 4D-CBCT images, may be used tocalculate d_(l) ^(m).

The predicted tumour position for the current frame, {circumflex over(x)}_(k|k-1), may then be estimated to be the prior displacement vectorbetween the previous and the current phase bin, d_(PhaseBin) _(k-1)^(PhaseBin) ^(k) , added on top of the previously tracked tumourposition, {circumflex over (x)}_(k-1|k-1):

{circumflex over (x)} _(k|k-1) ={circumflex over (x)} _(k-1|k-1) +d_(PhaseBin) _(k-1) ^(PhaseBin) ^(k) .  (1)

The d_(PhaseBin) _(k-1) ^(PhaseBin) ^(k) term incorporates the periodicnature of the respiratory motion into the prediction while the{circumflex over (x)}_(k-1|k-1) term takes the baseline shifts intoaccount. For the example illustrated in FIG. 2, the respiratory phasebin for each frame may be calculated retrospectively by the projectionintensity analysis method. In one example the projection intensityanalysis method is that of Kavanagh et al (Kavanagh A, Evans P M, HansenV N and Webb S 2009 Obtaining breathing patterns from any sequentialthoracic x-ray image set Phys. Med. Biol. 54(16), 4879).

Preferably, though not necessarily, the measure of the uncertainty ofthe first present dataset is a predicted covariance matrix of adistribution of the target's position. In some examples, the predictedcovariance matrix is a function of an uncertainty of the statetransition model (i.e. matrix Q_(k) below). The uncertainty of the statetransition model may be attributed to the respiratory-correlatedprediction model. In some examples, the uncertainty of the statetransition model is calculated using the target's positions tracked in apast time frame. In some examples, the past time frame spans from thepresent to ten seconds in the past. In other examples, other time framelengths may be selected (e.g. 1, 2, 3, or more seconds).

The use of a limited time frame for calculating, or estimating, theuncertainty of the state transition model is different from past methodswhich made use of all previously tracked positions. A limited, orshorter, time frame is important for cases where the motion patternchanges constantly.

The predicted covariance matrix, P_(k|k-1), is predicted from theprevious covariance update, P_(k-1|k-1), by:

P _(k|k-1) =F _(k-1) P _(k-1|k-1) F _(k-1) ^(T) +Q _(k)  (2)

Q_(k) is the prediction covariance matrix that describes the uncertaintyof the state transition model, which can be estimated by the covariancematrix of the distribution of previously tracked tumour positions. Insome examples, tracked tumour positions within the past ten seconds areempirically chosen to estimate Q_(k). In other examples, predictioncovariance matrix Q_(k) may be estimated using any number of pasttracked target positions. F_(k-1) is the Jacobian of the right-hand sideof Equation 1. Since the dr displacement vectors are calculated prior totreatment and are independent of tracking, F_(k-1) reduces to theidentity matrix I for all k. Equation 2 thus simplifies to:

P _(k|k-1) =P _(k-1|k-1) +Q _(k).  (3)

To initialise Equations 1 and 3 for the first tracking frame,{circumflex over (x)}_(0|0) may be set to the tumour position at anyarbitrary phase of the 4D-CBCT. Q₀ may be set to equal the covariancematrix of the distribution of tumour positions observed from all phasesof the 4D-CBCT.

In some examples, the state transition model predicts the targetposition to be the mean position within the same motion phase within thepast 10-140 seconds, and the uncertainty of the state transition modelis estimated to be the covariance matrix of the target positions withinthe same motion phase within the past 10-140 seconds. That is, in someexamples, the present position value of the target in the first presentdataset equals a mean position of the target within a same motion phaseof the target in the past 10 seconds to 140 seconds, and wherein theuncertainty of the state transition model is estimated to equal acovariance matrix of the target's positions within the same motion phasein the past 10 seconds to 140 seconds.

Step 130

Measurement of the second present dataset is performed on thetwo-dimensional image of the target. Measurement is done bytemplate-matching the two-dimensional image of the target with the modelof the target. In some examples, the two-dimensional image is processedprior to template-matching. Preferably, though not necessarily, atemplate-matching metric is calculated, or estimated, in measurementstep 130, either during or after template-matching has occurred. In someexamples, the template matching metric is the Mattes Mutual Information(MMI) metric of Mattes et al (Mattes D, Haynor D, Vesselle H, Lewellen Tand Eubank W 2001 Nonrigid multimodality image registration MEDICALIMAGING: 2001: IMAGE PROCESSING, PTS 1-3 4322, 1609-1620. MedicalImaging 2001 Conference, SAN DIEGO, Calif., Feb. 18-22, 2001). In otherexamples, the template matching metric is the Normalized CrossCorrelation (NCC) metric. In some examples, the measure of theuncertainty of the second present dataset is a function of thetemplate-matching metric.

In some examples, step 130 involves taking some form of measurement,z_(k), that can be related to the state variable byz_(k)=h_(k)(x_(k))+v_(k), where h_(k) (⋅) is the measurement responsefunction and v_(k) is the measurement noise. For kV-based tumourtracking, z_(k) is the 2D tumour position measured on the kV detectorplane, or on the imaging plane of the scanning device. h_(k) (⋅) is the3D-to-2D forward-projection transformation that converts a 3D positionin the patient coordinate space to its corresponding 2D position in thekV detector coordinate space for the current kV angle.

To measure z_(k), the tumour model of the current phase bin isforward-projected and aligned with the anatomy-subtractedtwo-dimensional image by template matching. The template matching isperformed using the Mattes Mutual Information (MMI) metric of Mattes etal (Mattes D, Haynor D, Vesselle H, Lewellen T and Eubank W 2001Nonrigid multimodality image registration MEDICAL IMAGING: 2001: IMAGEPROCESSING, PTS 1-3 4322, 1609-1620. Medical Imaging 2001 Conference,SAN DIEGO, Calif., Feb. 18-22, 2001), with a higher MMI value indicatinga better match. In some examples, a 20 mm by 20 mm search window centredat the forward-projected position of the predicted 3D tumour centroid isused. In other examples, other search window sizes may be used.

Preferably, the second present dataset further includes a measure of anuncertainty in the two-dimensional present position value of the target.In some examples, the measure of the uncertainty in the two-dimensionalpresent position value of the target is a function of thetemplate-matching metric.

The measure of the uncertainty of the measurement process, i.e. 2Dtemplate matching, is described by the 2-by-2 measurement covariancematrix, R_(k). In this work, R_(k) is estimated to be a diagonal matrix,because the template matching errors in the kV-lateral and kV-verticaldirections are generally independent of each other. For MMI valueshigher than a fixed threshold, both diagonal entries are set to thesquare of the kV projection pixel size. For MMI values lower than agiven threshold, both diagonal entries are set to be the square of thesize of the search window (e.g. 20 mm). In some examples, the value ofthe MMI threshold is empirically set to be 0.4. In other examples, theMMI threshold may be set to any other value, such as 0.1, 0.2, or 0.3.

The fact that the measure of the uncertainty of the measurement processtakes into account, or incorporates, the template-matching metric (e.g.the MMI value) is advantageous for markerless tumour tracking becausetemplate-matching in markerless tracking suffers from significantlylarger uncertainty than marker tracking due to the inferior visibilityof the tumour on 2D images (e.g. kV projections).

In some examples, template-matching is implemented, or run, through aGPU.

Steps 140 and 150

Step 140 combines the prediction and the measurement of steps 120 and130, respectively, to estimate both x and P according to the followingequations:

{circumflex over (x)} _(k|k) ={circumflex over (x)} _(k|k-1) +K _(k)[z_(k) −h _(k)(i _(kik-1))],  (4)

{circumflex over (P)} _(k|k)=(I−K _(k) H _(k))P _(k|k-1).  (5)

At step 150, equations 4 and 5 further update the values of the previousdataset whereby a next iteration (k+1) of method 100 will rely on{circumflex over (x)}_(k|k) and {circumflex over (P)}_(k|k) as theprevious dataset in prediction step 130.

The [z_(k)−h_(k)({circumflex over (x)}_(k|k-1))] term in Equation 4represents the discrepancy between the prediction and the measurement.The Kalman gain, K_(k), describes how much each component of theprediction state needs to be corrected towards the measurement, and iscalculated by:

K _(k) =P _(k|k-1) H _(k) ^(T)(H _(k) P _(k|k-1) H _(k) ^(T) +R_(k))⁻¹,  (6)

where H_(k) is the Jacobian of h_(k)(⋅). It can be seen from Equation 6that the Kalman gain takes into account the angular-dependent 3D-to-2Dgeometry via H_(k), the distribution of tumour 3D positions viaP_(k|k-1), and the reliability of the template matching result viaR_(k). The inclusion of P_(k|k-1) exploits the correlations betweenmotion components in different directions, making 3D tracking possibleeven though the measurement vector z_(k) contains only information onthe kV detector plane (i.e. 2D information of tumour position projectedon the kV detector plane).

The implementation of a Bayesian framework (or an extended Kalman filterbased on the same) for markerless tumour tracking is advantageous forseveral reasons. Firstly, conventional template-matching-based trackingmethods are rarely able to continuously track the tumour due to inferiortumour visibility. By combining motion prediction and measurementuncertainty estimation with template matching, method 100 is able totrack the tumour even when a good template match is not possible.Secondly, the use of the covariance matrix of tumour positiondistribution is advantageous for estimating the motion component alongthe measurement beam (e.g. X-ray or kV beam) direction, i.e. thedirection of the beam scanning device used to acquire thetwo-dimensional image of the body including the target, enabling 3Dtumour tracking using only 2D imaging. Although the example of FIG. 2illustrates an example implementation of method 100 for the purpose oftumour tracking, method 100 and the above framework may be used for any3D target localisation application that relies on 2D imaging.

In some examples, provisions may be taken to speed up the execution ofmethod 100. In some examples, the real-time respiratory phasecalculation method proposed by Ruan et al (Ruan D, Fessler J A, Balter JM and Keall P J 2009 Real-time profiling of respiratory motion: baselinedrift, frequency variation and fundamental pattern change Phys. Med.Biol. 54(15), 4777-4792) may be used in prediction step 120 to replaceretrospective phase calculation. In some examples, the processing of thetwo-dimensional image of the body including the target in step 110 (e.g.through method 400) and template-matching in step 130 may contribute toabout 95% of the computation time of method 100. Both of these steps arehighly parallelisable and can be significantly sped up through GPUimplementation.

In some examples, step 110 of processing the two-dimensional scannedimage of the body including the target and step 130 of measuring asecond present dataset of the target by template matching are executedin parallel. In some examples, step 110 of processing thetwo-dimensional scanned image of the body including the target and/orstep 130 of measuring a second present dataset of the target by templatematching are at least partially implemented through a GPU. In somecases, GPU implementation of steps 110 and 130 provides a computationtime of less than 1 second, which is advantageous for real-timeapplications. In some examples, step 120 is executed in parallel to step110. In some examples, step 120 is executed in parallel to step 130.

In some examples, method 100 is implemented using the Insight Toolkit(ITK) of Johnson et al (Johnson H J, McCormick M, Ibañez L andConsortium T I S 2015 The ITK Software Guide Kitware, Inc. ISBN1-930934-27-0) and/or the Reconstruction Toolkit (RTK) of Rit et al (RitS, Oliva M V, Brousmiche S, Labarbe R, Sarrut D and Sharp G C 2014 TheReconstruction Toolkit (RTK), an open-source cone-beam CT reconstructiontoolkit based on the Insight Toolkit (ITK) J. Phys.: Conf. Ser. 489(1),012079-012079).

In some examples, there is provided a method for three-dimensionaltracking of a target within a body. The method includes the step ofacquiring, by a scanning device, a two-dimensional image of the bodyincluding the target. The method further includes the step ofprocessing, by a processing system and/or a graphics processing unit,the two-dimensional image of the body including the target to obtain atwo-dimensional image of the target (i.e. a two-dimensional imageillustrating only the target). The method further includes the step ofpredicting, by the processing system, a first present dataset of thetarget by using a previous dataset of the target and a state transitionmodel, wherein the first present dataset includes at least athree-dimensional present position value of the target, and wherein theprevious dataset includes at least a three-dimensional previous positionvalue of the target. The method further includes the step of measuring asecond present dataset of the target by template-matching, by theprocessing system and/or the graphics processing unit, of thetwo-dimensional image of the target with a model of the target, whereinthe second present dataset includes at least a two-dimensional presentposition value of the target. The method further includes the step ofestimating, by the processing system, a third present dataset of thetarget by statistical inference using the first present dataset and thesecond present dataset, wherein the third present dataset includes atleast a three-dimensional present position value of the target. Themethod further includes the step of updating, by the processing system,the previous dataset of the target to match the third present dataset.

Referring to FIG. 5, there is illustrated an example system 500 forthree-dimensional tracking of a target located within a body, includinga scanning device 510 configured to acquire a two-dimensional scannedimage of the body including the target. System 500 further includes aprocessing system 520. Processing system 520 is configured to receiveand process the two-dimensional scanned image of the body including thetarget to obtain a two-dimensional image of the target. Processingsystem 520 is further configured to predict a first present dataset ofthe target by using a previous dataset of the target and a statetransition model. The first present dataset includes at least athree-dimensional present position value of the target. The previousdataset includes at least a three-dimensional previous position value ofthe target. Processing system 520 is further configured to measure asecond present dataset of the target by template-matching of thetwo-dimensional image of the target with a model of the target. Thesecond present dataset includes at least a two-dimensional presentposition value of the target. Processing system 520 further estimates athird present dataset of the target by statistical inference using thefirst present dataset and the second present dataset. The third presentdataset includes at least a three-dimensional present position value ofthe target. Processing system 520 further updates the previous datasetof the target to match the third present dataset.

Scanning device 510 may be an X-ray imaging device, or a kV imagingdevice, including an X-ray source and an X-ray detector. In someexamples, scanning device 510 is a CT scanner or a diagnostic imagingdevice. In some examples, scanning device 510 is any device able toacquire a two-dimensional, or projected, image of the target locatedwithin the body. Scanning device 510 may further include a detector fordetecting a surrogate signal. In some examples, scanning device 510 mayinclude one or more (e.g. two, three, four or more) of any of theabove-mentioned scanning devices. The scanned image of the bodyincluding the target may be an X-ray image, a kV image, a CT scan or anyother type of image or set of images illustrating the target within thebody, in accordance with scanning device 510.

Scanning device 510 may be in communication with processing system 520.Communication between scanning device 510 and processing system 520 mayoccur either through a wired or wireless connection. Each of scanningdevice 510 and processing system 520 may include further apparatusnecessary for communication (e.g. transmitter and receiver). Preferably,though not necessarily, system 500 further includes a communication link515 between scanning device 510 and processing system 520. Communicationlink 515 may be wired or wireless. Processing system 520 may be locatedwithin scanning device 510, or in proximity of scanning device 510, orremotely of scanning device 510. Communication link 515 allowstransmission of the scanned image from scanning device 510 to processingsystem 520. Communication link 515 may further allow for transmission ofany other data between scanning device 510 and processing system 520(e.g. data from a surrogate signal).

In some examples, the two-dimensional scanned image of the bodyincluding the target illustrates anatomy of the body in the vicinity ofthe target. In those examples, in order to obtain the two-dimensionalimage of the target (excluding the body anatomy), processing system 520may process the two-dimensional scanned image to remove the images ofthe body anatomy and isolate the image of the target. To this end,processing system 520 may project a model of the body anatomy excludingthe target to align with the body anatomy of the two-dimensional scannedimage and subtract the projected model of the body anatomy from thetwo-dimensional scanned image.

In some examples, processing system 520 further includes a graphicsprocessing unit (GPU) or a visual processing unit (VPU). The GPU may beused to accelerate the measurement of the second present dataset byaccelerating the template-matching process. In some examples, processingsystem 520 comprises a memory unit. The memory unit may store the modelof the target and/or the model of the body anatomy excluding the target.

Referring to FIG. 6, there is illustrated an example processing system520. In particular, the processing system 520 generally includes atleast one processor 602, or processing unit or plurality of processors,memory 604, at least one input device 606 and at least one output device608, coupled together via a bus or group of buses 610. In certainembodiments, input device 606 and output device 608 could be the samedevice. An interface 612 can also be provided for coupling theprocessing system 520 to one or more peripheral devices, for exampleinterface 612 could be a PCI card or PC card. At least one storagedevice 614 which houses at least one database 616 can also be provided.The memory 604 can be any form of memory device, for example, volatileor non-volatile memory, solid state storage devices, magnetic devices,etc. The processor 602 could include more than one distinct processingdevice, for example to handle different functions within the processingsystem 600.

Input device 606 receives input data 618 and can include, for example, adata receiver or antenna such as a modem or wireless data adaptor, dataacquisition card, etc. Input data 618 could come from different sources,for example scanning device 510 and/or with data received via a network.Output device 608 produces or generates output data 620, for examplerepresenting position values, two-dimensional images and/orthree-dimensional images, and can include, for example, a display deviceor monitor in which case output data 620 is visual, a printer in whichcase output data 620 is printed, a port for example a USB port, aperipheral component adaptor, a data transmitter or antenna such as amodem or wireless network adaptor, etc. Output data 620 could bedistinct and derived from different output devices, for example a visualdisplay on a monitor in conjunction with data transmitted to a network.A user could view data output, or an interpretation of the data output,on, for example, a monitor or using a printer. The storage device 614can be any form of data or information storage means, for example,volatile or non-volatile memory, solid state storage devices, magneticdevices, etc.

In use, the processing system 520 is adapted to allow data orinformation to be stored in and/or retrieved from, via wired or wirelesscommunication means, the at least one database 616. The interface 612may allow wired and/or wireless communication between the processingunit 602 and peripheral components that may serve a specialised purpose.The processor 602 receives instructions as input data 618 via inputdevice 606 and can display processed results or other output to a userby utilising output device 608. More than one input device 606 and/oroutput device 608 can be provided. It should be appreciated that theprocessing system 520 may be any form of terminal, server, specialisedhardware, or the like.

In some examples, system 500 is configured to implement method 100.

System 500 may further include an apparatus for interacting with, oroperating on, the target. Referring to FIG. 7, there is illustrated asystem 700 for tracking and interacting with a target located within abody. System 700 includes a scanning device 710, a processing system720, and an apparatus 730. Scanning device 710 acquires atwo-dimensional scanned image of the body including the target.Processing system 720 receives the two-dimensional scanned image andexecutes, or implements, method 100, outputting a third present datasetthat includes at least a three-dimensional present position value of thetarget. Apparatus 730 interacts with, or operates on the target and isadjusted, or its operation is adjusted, in response to, or based on, thethird present dataset. In some examples, system 700 may further includeone or more controllers for adjusting the operation of apparatus 730.

In some examples, where the target is a tumour or lung tumour, system700 is a radiotherapy system. In some examples, apparatus 730 is atreatment apparatus. In some examples, apparatus 730 is a therapeuticradiation source. In some examples, the direction of radiation output isadjusted in response to, or based on, the third present dataset. In someexamples, apparatus 730 is a patient table. In some examples, theposition and/or orientation of the table is adjusted in response to, orbased on, the third present dataset. In other examples, apparatus 730 isany type of device that is part of a radiotherapy system and whoseoperation would need to be adjusted in response to tumour motion.

With reference to FIG. 8, there is illustrated an example radiotherapysystem 800 for tracking and treating a tumour located within a body(e.g. patient 801). Radiotherapy system 800 includes a scanning device810, a processing system 820, a therapeutic radiation source 831, and apatient table or couch 833. Any one or more of scanning device 810,processing system 820, radiation source 831, and patient table 833 maybe integrated, or in-built, into a radiotherapy system (i.e. aradiotherapy machine), or they may be provided as separate elements inradiotherapy system 800.

During treatment, scanning device 810 continuously, or repeatedly,acquires two-dimensional kV X-ray scanned images of the patient andsends each scanned image (or a processed version of said scanned image)to processing system 820. Processing system 820 executes method 100 foreach scanned image received, and estimates a third present dataset,which includes a three-dimensional position of the tumour.

In some examples, processing system 820 is configured to adjustradiation source 831 and/or patient table 833 in response to thetumour's motion indicated by the third present dataset. In otherexamples, the estimated third present dataset is sent, transmitted, ortransferred to another processing system (not shown) or to a humanoperator who may then determine how to adjust radiation source 831and/or patient table 833.

In some examples, adjustment of radiation source 831 may includeadjustment of the direction of radiation output to redirect a treatmentbeam to follow the tumour motion. In some examples, adjustment ofradiation source 831 may further include pausing, stopping, or reducingradiation output if the tracked three-dimensional position of the tumourexceeds a certain prescribed range, and resuming the treatment beam whenthe three-dimensional position of the tumour returns within theprescribed range. In some examples, adjusting patient table 833 includesadjusting a position and/or orientation of patient table 833 to maintainthe tumour position in a path of therapeutic radiation from radiationsource 831.

Adjustment of radiation source 831 and/or patient table 833 may be doneby controllers (not shown). In some examples the controllers areelectrical controllers. In some examples, the controllers are mechanicalcontrollers. The controllers may be electrically or mechanicallycontrolled by processing system 820, or by any other processing system,or by a human operator in response to the third present datasetestimated through method 100 or by system 800.

Adjustment of a treatment apparatus (e.g. a therapeutic radiation sourceand/or a patient table) in response to tumour motion tracked by method100 or by system 700 is advantageous since it enables continuous,uninterrupted treatment. In some cases, adjustment of treatmentapparatus further enables tailoring of treatment to suit the conditionsof a moving tumour, and it may further minimise radiation damage totissue surrounding the tumour. These advantages provide an improvedtreatment method and/or system due to reduce treatment time and recoverytime.

FURTHER EXAMPLES

The following examples provide more detailed discussion of particularembodiments. The examples are intended to be merely illustrative and notlimiting to the scope of the present invention.

Example 1: Patient Data

The proposed method was retrospectively validated on a total of 13clinical cases from two different sets of patient data:

-   -   (i) The CBCT cases (11 cases): CBCT projections from an        NCI-sponsored trial with locally advanced lung cancer patients.        This dataset was included and described in a previous        publication (Shieh C C, Keall P J, Kuncic Z, Huang C Y and Feain        I 2015 Markerless tumor tracking using short kilovoltage imaging        arcs for lung image-guided radiotherapy Phys. Med. Biol. 60(24),        9437). More details on the datasets can be found in Roman et al        (Roman N O, Shepherd W, Mukhopadhyay N, Hugo G D and Weiss E        2012 Interfractional positional variability of fiducial markers        and primary tumors in locally advanced non-small-cell lung        cancer during audiovisual biofeedback radiotherapy Int. J.        Radiat. Oncol. 83(5), 1566-1572).    -   (ii) The SABR cases (2 cases): kV images during MV beam-on from        a lung SABR trial (NCT02514512).

The CBCT cases included 11 CBCT scan pairs from four locally advancedlung cancer patients with central tumours, which often suffer frominferior adjacent contrast on kV images due to being attached to themediastinum and are challenging to track. The sizes of the tumoursranged from 30.2 cm³ to 88.9 cm³. Each patient was implanted with 2-4fiducial markers around the tumour, the trajectories of which were usedas ground truths to quantify the errors of markerless tumour trackingimplemented through method 100.

Each CBCT scan pair contained two CBCT scans that were acquired withinthe same day. The first scan in the pair was used as the pre-treatmentCBCT to build the tumour and body anatomy models, while markerlesstumour tracking was performed on the second scan. The tumour positionsbetween some of the scan pairs were misaligned as the time gap betweenthe two scans ranged from a half to a couple of hours.

To simulate pre-treatment patient alignment, the pre-treatment CBCT aswell as the tumour and body anatomy models were rigidly shifted to alignwith the mean tumour position within the first 10 seconds of the secondscan. Markers were removed from the projection images to avoid biasingthe markerless tracking results. Each CBCT scan contained either 1200 or2400 half-fan projections acquired with a frame rate of about 5 Hz fromthe Varian on-board kV imaging device (Varian Medical Systems, PaloAlto, Calif.). The size of each projection was 1024 pixels by 768pixels, with a pixel spacing of 0.388 mm. The pre-treatment 4D-CBCT (10bins) was reconstructed using the anatomical-adaptive imageregularisation (AAIR) technique (Shieh C C, Kipritidis J, O'Brien R T,Cooper B J, Kuncic Z and Keall P J 2015 Improving thoracicfour-dimensional cone-beam CT reconstruction with anatomical-adaptiveimage regularization (AAIR) Phys. Med. Biol. 60(2), 841) combined withthe prior-image-constrained-compressed-sensing (PICCS) algorithm (Chen GH, Tang J and Leng S 2008 Prior image constrained compressed sensing(PICCS): A method to accurately reconstruct dynamic CT images fromhighly undersampled projection data sets Med. Phys. 35(2), 660-663).Audiovisual biofeedback breathing guidance was performed during both CTand CBCT acquisitions.

Two patients (one fraction each) from a lung SABR trial (NCT02514512)designed to investigate the benefits of real-time adaptive treatment andkV imaging were included. The SABR cases represent realistic clinicalscenarios for validating markerless tumour tracking for two mainreasons. Firstly, the in-treatment kV images for tracking were acquiredwith the presence of MV scatter noise. Secondly, the pre-treatment CBCTscan was acquired with a standard protocol of one-minute scan time andaround 680 half-fan projections. The first patient had a 7.4 cm³ tumourattached to the chest wall, while the second patient had a 13 cm³ tumournear the diaphragm. Each patient was implanted with threeelectromagnetic transponder beacons, the motions of which were sent tothe multileaf-collimator to enable real-time treatment beam adaptation.The trajectories of the beacons were used as the ground truths toquantify the errors of markerless tumour tracking. The patients weretreated on a Varian Triology (Varian Medial Systems, Palo Alto, Calif.).A one-minute pre-treatment CBCT scan was acquired for each fraction, andwas reconstructed into a 10-bin 4D-CBCT using the PICCS algorithm (see,for example, Chen G H, Tang J and Leng S 2008 Prior image constrainedcompressed sensing (PICCS): A method to accurately reconstruct dynamicCT images from highly undersampled projection data sets Med. Phys.35(2), 660-663) for building the tumour and anatomic models. Prior totreatment the patient was shifted such that the tumour was around theisocenter. The same shift was applied to the tumour and anatomic models.kV images were continuously acquired during treatment with a frame rateof about 5 Hz and pixel spacing of 0.388 mm. The field of view of the kVimages was cropped to approximately 20 cm by 20 cm to reduce imagingdose. Beacons were removed from the kV images to avoid biasing themarkerless tracking results. Audiovisual biofeedback breathing guidancewas performed during CT, CBCT, and treatment.

Example 1: Electrochemical Measurements

The ground truth for evaluating tracking error was built from thetrajectory of the fiducial marker (CBCT cases) or the electromagneticbeacon (SABR cases) that was closest to the tumour centroid. Thetrajectories of the fiducial markers were obtained using a robusttemplate-based segmentation method (Poulsen P R, Fledelius W, Keall P J,Weiss E, Lu J, Brackbill E and Hugo G D 2011 A method for robustsegmentation of arbitrarily shaped radiopaque structures in cone-beam CTprojections Med. Phys. 38(4), 2151-2156) and aprobability-density-function based on 2D-to-3D conversion (Poulsen P R,Cho B and Keall P J 2008 A method to estimate mean position, motionmagnitude, motion correlation, and trajectory of a tumor from cone-beamCT projections for image-guided radiotherapy Int. J. Radiat. Oncol.72(5), 1587-1596). The trajectories of the electromagnetic beacons wererecorded by the Calypso Tracking System (Varian Medical Systems, PaloAlto, Calif.).

To relate marker/beacon positions to tumour position, areference-to-tumour vector is calculated for each respiratory phase binof the pre-treatment 4D-CBCT as the displacement from the meanmarker/beacon centroid to the tumour centroid. The ground truth positionat imaging frame k, g_(k), is then the trajectory of the marker/beaconcentroid plus the reference-to-tumour vector of the corresponding phasebin at frame k.

The tracking error at frame k, e_(k), was defined by:

e _(k) ^(Tracking) ={circumflex over (x)} _(k|k) −g _(k),  (7)

where {circumflex over (x)}_(k|k) is the 3D tumour position as estimatedby the proposed markerless tumour tracking method 100. For each trackingfraction, the means and standard deviations of the left-right (LR),superior-inferior (SI), and anterior-posterior (AP) components of e_(k)^(Tracking)'s were calculated. The mean and standard deviation of the 3Derror, ∥e_(k) ^(Tracking)∥, was also calculated.

To compare markerless tumour tracking with the current standard of care,i.e. a single estimation of tumour position based on the pre-treatment3D CBCT, the standard-of-care error was defined by:

e _(k) ^(Standard) =x _(3DCBCT) −g _(k),  (8)

where x_(3DCBCT) is the tumour position estimated from the pre-treatment3D CBCT. Similarly to e_(k) ^(Tkracking), the mean and standarddeviation of e_(k) ^(Standard)'s in each direction as well as its 3Dnorm were calculated for every fraction.

Example 1: Results

For imaging frames where the tumour can be visually identified, visualinspection suggested that the proposed markerless tumour tracking methodwas able to continuously track the tumours at every imaging angle forall 13 cases investigated. The mean and standard deviation of the 3Dtracking error ranged from 1.55-2.88 mm and 0.63-1.46 mm, respectively.The 95th percentile of the 3D tracking error ranged from 2.62-5.77 mm.The 5th-to-95th percentile motion range observed from the ground truthswas 1.38-5.51 mm for LR, 4.40-15.26 mm for SI, and 1.99-7.91 mm for AP.

FIG. 9 illustrates the markerless tumour tracking trajectories of caseswith the lowest 3D tracking error, where the mean values of the LR, SI,and AP trajectories were shifted for display to 10 mm, 0 mm, and −10 mm.FIG. 10 illustrates the markerless tumour tracking trajectories of caseswith the highest 3D tracking errors, where the mean values of the LR,SI, and AP trajectories were shifted for display to 20 mm, 0 mm, and −20mm.

The lowest mean 3D tracking error was found for the first scan pair ofpatient 2 of the CBCT cases, with a mean 3D error of 1.55±0.63 mm (shownin FIG. 9). Overall the tracking trajectory agreed closely with theground truth, except for around t=60 s, where LR errors of about 3 mmwere observed. The highest mean 3D tracking error was found for patient2 of the SABR cases, with a mean 3D error of 2.88±1.43 mm (shown in FIG.10). The larger tracking errors are likely attributed to the presence ofMV scatter in the kV images and the lower quality of the tumour andanatomic models due to the limited amount of pre-treatment CBCTprojections. The tumour motion range was also considerably larger forthis case. Nevertheless, even in this challenging case the pattern ofthe motion was consistently tracked at every imaging angle.

FIG. 11 illustrates a comparison of the mean tracking errors in LR, SI,and AP to the standard-of-care errors for all 13 cases investigated. Thestandard deviations of the errors were plotted as error bars. Casesmarked with asterisks indicate that the proposed markerless tumourtracking method resulted in significantly smaller errors than thestandard of care.

The mean tracking errors were always closer to 0 mm than the meanstandard-of-care errors in every direction, indicating the ability ofthe proposed method to track baseline shifts. The proposed markerlesstumour tracking approach was found to perform the best in the SIdirection. Tracking errors were significantly smaller than thestandard-of-care errors for all cases in the SI direction (p-value lessthan 0.02 for patient 4 scan 1 of the CBCT cases and p-value less than10⁻⁴ for all other cases), while only for 10 and 6 cases in the LR andAP directions, respectively (p-value less than 10).

To investigate the benefits to patients of markerless tumour trackingover the standard of care, FIG. 12 compares the margins required with orwithout tracking to cover 95% of tumour motion in the LR, SI, and APdirections. Markerless tumour tracking always resulted in smallermargins in the SI directions with one exception.

Considerable reduction in SI margin was found for cases with greaterthan 10 mm 5th-to-95th SI motion range. For patient 2 of the SABR cases,the reduction in SI margin was as large as 9.5 mm (from 13.9 mm to 4.4mm). Margin reduction in the LR and AP directions was less pronounced.Generally with markerless tumour tracking, a margin of 3 mm, 6 mm, and4.5 mm in the LR, SI, and AP directions were sufficient to encompass 95%of tumour motion for all 13 cases investigated.

Results on the SABR cases demonstrated the ability of the proposedapproach to handle practical challenges such as MV scatter and low CBCTimage quality, and thus its practical feasibility. The benefits topatients of markerless tumour tracking over the standard of care washighlighted for patients with greater than 10 mm 5th-to-95th SI motionranges, with up to 9.5 mm reduction in SI margin. The clinicalimplementation of the proposed method enables more accurate and preciselung radiotherapy using existing hardware and workflow.

There are however a number of limitations in the example implementationof method 100. A first limitation is that, while fiducial marker orbeacon motions were used as the ground truths, these can havedifferential motions with the tumour. Hardcastle et al (Hardcastle N,Booth J, Caillet V, O'Brien R, Haddad C, Crasta C, Szymura K and Keall P2016 Electromagnetic beacon insertion in lung cancer patients andresultant surrogacy errors for dynamic MLC tumour tracking The 58thAnnual Meeting of the American Association of Physicists in Medicine,Washington D.C.) has reported 0-3 mm deviations between beacon andtumour motion. Another limitation is the inferior quality of the tumourand anatomic models for the SABR cases due to the challenge inreconstructing 4D-CBCT images using a one-minute scan. In practice,other improved algorithms for iterative 4D-CBCT reconstruction (see, forexample, Schmidt M L, Poulsen P R, Toftegaard J, Hoffmann L, Hansen Dand Srensen T S 2014 Clinical use of iterative 4D-cone beam computedtomography reconstructions to investigate respiratory tumor motion inlung cancer patients Acta Oncol. 53(8), 1107-1113. PMID: 24957556. URL:http://dx.doi.org/10.3109/0284186X.2014.927585) will likely furtherimprove the performance of the proposed method for cases with one-minutepre-treatment CBCT scans as the prior images.

Optional embodiments may also be said to broadly include the parts,elements, steps and/or features referred to or indicated herein,individually or in any combination of two or more of the parts,elements, steps and/or features, and wherein specific integers arementioned which have known equivalents in the art to which the inventionrelates, such known equivalents are deemed to be incorporated herein asif individually set forth.

Although a preferred embodiment has been described in detail, it shouldbe understood that many modifications, changes, substitutions oralterations will be apparent to those skilled in the art withoutdeparting from the scope of the present invention.

Throughout this specification and the claims which follow, unless thecontext requires otherwise, the word “comprise”, and variations such as“comprises” or “comprising”, will be understood to imply the inclusionof a stated integer or step or group of integers or steps but not theexclusion of any other integer or step or group of integers or steps.

1-52. (canceled)
 53. A method for three-dimensional tracking of a targetlocated within a body, the method performed by at least one processingsystem, and comprising the steps of: processing a two-dimensionalscanned image of the body including the target to obtain atwo-dimensional image of the target, wherein the two-dimensional scannedimage is acquired from a scanning device; predicting a first presentdataset of the target by using a previous dataset of the target and astate transition model, wherein the first present dataset includes atleast a three-dimensional present position value of the target, andwherein the previous dataset includes at least a three-dimensionalprevious position value of the target; measuring a second presentdataset of the target by template-matching of the two-dimensional imageof the target with a model of the target, wherein the second presentdataset includes at least a two-dimensional present position value ofthe target; estimating a third present dataset of the target bystatistical inference using the first present dataset and the secondpresent dataset, wherein the third present dataset includes at least athree-dimensional present position value of the target; and updating theprevious dataset of the target to match the third present dataset. 54.The method of claim 53, wherein the statistical inference utilizes anextended Kalman filter.
 55. The method of claim 53, wherein the model ofthe target provides information of three-dimensional spatialdistribution of the target at one or more points in time.
 56. The methodof claim 53, wherein the model of the target is derived from one or moreprior images.
 57. The method of claim 56, wherein the model of thetarget is further derived from a surrogate signal indicative of a motionof the target.
 58. The method of claim 53, wherein each present datasetof the target further includes a measure of an uncertainty in therespective position value of the target.
 59. The method of claim 58,wherein the measure of the uncertainty of the first present dataset is apredicted covariance matrix of a distribution of the target position.60. The method of claim 59, wherein the predicted covariance matrix is afunction of an uncertainty of the state transition model.
 61. The methodof claim 60, wherein the uncertainty of the state transition model iscalculated using a target position tracked in a past time frame.
 62. Themethod of claim 60, wherein the present position value of the target inthe first present dataset equals a mean position of the target within asame motion phase of the target in the past 10 seconds to 140 seconds.63. The method of claim 62 further wherein the uncertainty of the statetransition model is estimated to equal a covariance matrix of the targetpositions within the same motion phase in the past 10 seconds to 140seconds.
 64. The method of claim 53, wherein measuring the secondpresent dataset further comprises processing the two-dimensional imageof the target prior to template-matching.
 65. The method of claim 53,wherein measuring the second present dataset further includescalculating a template-matching metric.
 66. The method of claim 53,wherein the second present dataset further includes a measure of anuncertainty in the two-dimensional present position value of the target.67. The method of claim 66, wherein the measure of the uncertainty inthe two-dimensional present position value of the target is a functionof the template-matching metric.
 68. The method of claim 53, wherein thetarget is a tumor.
 69. The method of claim 53, wherein the target is alung tumor.
 70. The method of claim 53, wherein the model of the targetaccounts for the periodic nature of tumor motion.
 71. The method ofclaim 53, wherein predicting the first present dataset of the targetfurther comprises accounting for the periodic nature of tumor motion.72. The method of claim 53, wherein the body is a human body.